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(54) Humanoid robot communicating with body language 



(57) A legged mobile robot realizes a motion lan- 
guage by a time-series change in an actuator angle or 
a motion pattern using the four limbs and the trunk. A 
motion language which includes, for example, a motion 
pattern which is an approximation of the contour/shape 
of a character is used, so that even a robot or a human 
being which does not possess the same motion lan- 
guage database can determine the meaning and char- 
acter which is indicated by each motion pattern as a re- 
sult of visually recognizing and interpreting the contour/ 
form which is indicated by each motion pattern. For ex- 
ample, a robot which has stepped into a dangerous 
working area can give a message concerning, for exam- 
ple, the condition of the working area to an observer at 
a remote location without using any data communica- 
tions means. The legged mobile robot communicates by 
moving the limbs and/or the trunk. 
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Description 



[0001 ] The present invention relates to articulated ro- 
bots and methods of controlling articulated robots. 
[0002] A robot is a mechanical device which emulates 
the motion of a human being by making use of electrical 
and magnetic actions. The term robot is said to be de- 
rived from the Slavic word ROBOTA (slavish machine). 
In our country, the widespread use of robots began from 
the end of the 1 960s, many of which were industrial ro- 
bots, such as manipulators and conveyance robots, 
used, for example, for the purpose of achieving auto- 
matic industrial operations in factories without humans 
in attendance. 

[0003] Installation-type robots, such as arm-type ro- 
bots, which are provided so as to be implanted at a par- 
ticular place function only in fixed/localized working 
spaces for carrying out parts assembly/sorting and the 
like. In contrast, mobile robots can function in unlimited 
working spaces, so that they can freely move in prede- 
termined paths or pathless areas in order to perform in 
place of human beings a predetermined or a desired hu- 
man task, or to provide various wide-ranging services 
in place of living beings such as humans and dogs. Of 
the mobile robots, legged mobile robots are unstable 
compared to crawler-type robots and wheeled robots, 
so that it becomes difficult to control their posture and 
walking. However, legged mobile robots are excellent 
robots in that they can move up and down steps and 
ladders, go over obstacles, and walk/run flexibly regard- 
less of whether the ground is leveled or unleveled. 
[0004] In recent years, advances have been made in 
the research and development of legged mobile robots 
such as pet robots and humanoid robots, with pet robots 
emulating the mechanism of the body of an animal 
which moves using four legs, such as a dog or a cat, 
and the motion thereof, and with humanoid robots being 
designed using as a model the mechanism of the body 
of an animal which moves erect using two legs, such as 
a human being, and the motion thereof. Therefore, there 
is increasing expectations for putting such robots into 
practical use. 

[0005] The significance of carrying out research and 
development on legged mobile robots called humanoid 
robots can be understood from, for example, the follow- 
ing two viewpoints. 

[0006] The first viewpoint is related to human science. 
More specifically, through the process of making a robot 
having a structure which is similar to that having lower 
limbs and/or upper limbs of human beings, thinking up 
a method of controlling the same, and simulating the 
walking of a human being, the mechanism of the natural 
motion of a human being, such as walking, can be er- 
gonomically understood. The results of such research 
can considerably contribute to the development of other 
various research fields which treat human motion mech- 
anisms, such as ergonomics, rehabilitation engineering, 
and sports science. 



[0007] The other viewpoint is related to the develop- 
ment of practical robots as partners of human beings 
which help them in life, that is, help them in various hu- 
man activities in situations in living environments and in 
5 various other situations in everyday life. Functionally, in 
various aspects of the living environment of human be- 
ings, these robots need to be further developed by learn- 
ing methods of adapting to environments and acting in 
accordance with human beings which have different 
io personalities and characters while being taught by hu- 
man beings. Here, it is believed that making the form 
and structure of the robot the same as those of a human 
being is effective in achieving smooth communication 
between the robot and a human being. 
15 [0008] For example, when teaching a robot a way of 
passing through a room by avoiding obstacles which 
should not be. stepped on, it is much easier for the user 
(worker) to teach it to a robot which walks using two legs 
having the same form as the user than a crawler-type 
20 or a robot which walks using four legs having a com- 
pletely different structure from the user. In this case, it 
must also be easier for the robot to learn it (Refer to, for 
example, "Controlling a Robot Which Walks Using Two 
Legs" by Takanishi (Jidosha Gijutsukai Kanto Shibu 
25 <Koso> No. 25, April, 1 996)). 

[0009] The working space and living space of human 
beings are formed in accordance with the behavioral 
mode and the body mechanism of a human being which 
walks erect using two legs. In other words, for moving 
3 o present wheeled mechanical systems or other such 
driving devices as moving means, the living space of 
human beings has too many obstacles. It is preferable 
that the movable range of the robot be about the same 
as that of human beings in order for the mechanical sys- 
35 tern, that is, the robot to carry out various human tasks 
in place of them, and to deeply penetrate the living 
space of human beings. This is the reason why there 
are great expectations for putting a legged mobile robot 
into practical use. In order to increase the capability of 
40 the robot to exist harmoniously in the living environment 
of human beings, it is essential for the robot to possess 
a human form. 

[001 0] Humanoid robots can be used to carry out var- 
ious difficult operations, such as in industrial tasks or 
« production work, in place of human beings. They carry 
out in place of human beings dangerous or difficult op- 
erations at places where human beings cannot easily 
step into, such as maintenance work at nuclear power 
plants, thermal power plants, or petrochemical plants 
50 parts 

transportation/assembly operations in manufacturing 
plants, cleaning of tall buildings, rescuing of people at 
places where there is a fire, and the like. 
[001 1] Humanoid robots can also be used "to live to- 
55 gether" in the same living space as human beings, that 
is, for entertainment purposes. In this application, the 
robot is characterized more as being closely related to 
life rather than as helping human beings in life such as 
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performing tasks in place of them. 
[0012] The theme of entertainment robots is the re- 
search and development of the motion patterns which 
are executed during a task rather than realizing specific 
industrial purposes at high speeds and with high preci- 5 
sion. in other words, it is preferable for the entertainment 
robots to faithfully reproduce the mechanism which al- 
lows harmonious movement of the whole body pos- 
sessed by nature by animals which walk erect using two 
legs, such as human beings and monkeys, and to make 10 
them realize this natural and smooth movement. In em- 
ulating a highly intelligent animal which stands erect, 
such as human beings and monkeys, it is to be consid- 
ered that a motion pattern using the four limbs is natural 
for a living body, and that it is desirable that the motions 15 
are sufficiently indicative of emotions and feelings. 
[0013] Further, entertainment robots are required not 
only to faithfully execute a previously input motion pat- 
tern, but also to express its feelings by moving in a lively 
manner in response to the words and actions of a person 
(such as speaking highly of someone, scolding some- 
one, or hitting someone). In this sense, entertainment 
robots which emulate human beings are rightly called 
humanoid robots. 

[0014] In conventional toy machines, the relationship 
between the operations which are carried out by the us- 
er and the response motion is fixed/standardized, so 
that the same motion is merely repeated, causing the 
user to eventually get tired of the toy machines. In con- 
trast, entertainment robots, though they move in accord- 
ance with a motion generation time series model, can 
change this time series model, that is, impart a learning 
effect, in response to the detection of an external stim- 
ulus which is produced by, for example, the operation of 
the user. Therefore, the relationship between the oper- 
ations which are carried out by the user and the re- 
sponding motion is programmable, making it possible to 
provide a motion pattern which does not make the user 
tired of it or which conforms to the preferences of the 
user. In addition, by operating the robot, the user can 
enjoy a type of educational simulation. 
[0015] A robot is made to operate to, for example, per- 
form certain "tasks" such as parts assembly, or to 
"move" to, for example, transport things. Therefore, the 
robot is basically a means for achieving a specific result. 
[0016] In entertainment robots where human beings 
and pets are used as models, the word "task" has a wid- 
er meaning, so that, in addition to including industrial 
and production activities, it also includes the execution 
of entertainment motion patterns such as "dancing" and 
"greeting someone." Further, the range of tasks using 
the legs extend from those including motion patterns 
which are executed merely for the purpose of "moving" 
from place to place to those which include motion pat- 
terns which involve high-speeds and are strongly char- 
acterized as entertainment motion patterns, such as 
running and jumping. 

[0017] However, these tasks which are executed by 



entertainment robots are similarly confined to being car- 
ried out to achieve particular results. 
[001 8] Other than for purposes for achieving a certain 
result, human beings move for conversational purpos- 
es, such as making gestures and waving the hands. 
Such conversational motions make it possible for hu- 
man beings to understand each other using the sense 
of sight. The observer of the motion pattern can intui- 
tively understand the meaning of the motion pattern of 
the other person, thereby making it unnecessary to com- 
municate through words or to interpret words. For ex- 
ample, even if the two people use different languages, 
they can understand each other's intentions using mo- 
tion patterns even if they do not understand each other's 
language very well. In other words, communication by 
a motion may at times make it possible for people to 
smoothly understand each other. 
[0019] In contrast to this, a robot is basically a me- 
chanical device, and can communicate with external de- 
vices at high speeds and with high precision through a 
predetermined communications medium by following a 
predetermined communications protocol. The term ex- 
ternal device used here may be another robot or a host 
computing machine system for generating a command 
to a robot or for controlling the motion of the robot. In 
recent years, data communications technologies by 
wire or by radio have improved, so that one or more 
communications interfaces can be installed in the robot 
device. 

[0020] However, robots cannot be made to under- 
stand each other unless the proper communications in- 
terface or protocols are used. 
[0021 ] compared to other types of robots such as arm 
robots and crawler robots, robots which having four rel- 
atively long limbs, such as humanoid robots, have a 
much higher degree of freedom provided by joints. How- 
ever, this high degree of freedom has not been positively 
made use of in realizing communication by a motion of 
the robot 100. 

[0022] Various aspects and features of the present in- 
vention are defined in the appended claims. 
[0023] In view of the above-described problems, ac- 
cording to a first aspect of the present invention, there 
is provided an articulated robot which comprises a plu- 
rality of joints. 

The articulated robot includes a movement controller for 
controlling the movement of each of the joints, a motion 
language accumulator for accumulating a motion lan- 
guage formed by a motion pattern of the joints, and a 
movement output controller for giving the motion lan- 
guage to be used to the movement controller as a result 
of taking the motion language to be used from the mo- 
tion language accumulator. 

The motion patterns are executed to express the inten- 
tions and details of communication. 
[0024] Embodiments of the present invention relate to 
an articulated robot like a legged robot having at least 
limbs and a trunk, and a method of controlling the motion 
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of the articulated robot, and, more particularly, to a leg- 
ged robot which executes various motion patterns by us- 
ing the limbs and/or the trunk, and a method of control- 
ling the motion of the legged robot. 
[0025] Still more particularly, embodiments of the 
present invention relate to a legged robot which com- 
municates by moving the limbs and/or the trunk, and 
methods of controlling the motion of the legged robot. 
Even still more particularly, embodiments of the present 
invention relate to a legged robot which comes to an un- 
derstanding through various motion patterns making 
use of the limbs and/or the trunk, and methods of con- 
trolling the motion of the legged robot. 
[0026] Accordingly, embodiments of the present in- 
vention can provide an excellent or at least improved 
legged robot comprising at least limbs and a trunk, and 
a method of controlling the motion of the legged robot. 
[0027] Embodiments of the present invention can also 
provide an excellent or at least improved legged robot 
which can execute various motion patterns using the 
limbs and/or the trunk, and a method of controlling the 
motion of the legged robot. 

[0028] Embodiments of the present invention can also 
provide an excellent or at least improved legged robot 
which can communicate by moving the limbs and/or the 
trunk, and a method of controlling the motion of the leq- 
ged robot. 

[0029] Embodiments of the present invention can also 
provide an excellent or at least improved legged robot 
which through various motion patterns using the limbs 
and/oir the trunk can come to an understanding with the 
external world such as robots which have not been con- 
nected through data transmission media, and a method 
of controlling the motion of the legged robot. 
[0030] In one form of the first aspect of the present 
invention, the articulated robot further includes an ex- 
ternal interface used for performing operations including 
an image input operation, a sound input/output opera- 
tion, and a data transmission/reception operation 
through a communications medium, and a communica- 
tion controller for controlling the communication based 
on input/output data through the external interface. In 
the articulated robot, the communication controller de- 
termines the motion language to be used in accordance 
with the details of the communication. 
[0031] In anotherform of the first aspect of the present 
invention, the motion language is realized using a meth- 
od which forms a sentence by repeating the formation 
of a plurality of characters with the basic mode of for- 
mation being in character units, or by using a method 
which realizes the communication by a motion which 
has a meaning or based on a time-series chanqe in a 
joint angle. 

[0032] In still another form of the first aspect of the 
present invention, a plurality of motion languages are 
combined to express the intentions of the robot or the 
details of the communication. In addition, the motion lan- 
guage accumulator includes a motion pattern used to 



direct the starting of the realization of the motion lan- 
guage, and/or a motion pattern used to direct the ending 
of the realization of the motion language. For example, 
the starting and the ending of the realization of the mo- 
$ tion language can be indicated by the raising of the right 
hand and the left hand of the humanoid robot. By exe- 
cuting the motion patterns thereof in starting and ending 
the communication, the observer of the motion lan- 
guage can easily and reliably recognize the motion lan- 
fo guage. 

[0033] In still another form of the first aspect of the 
present invention, when a plurality of motion languages 
are combined to express the intentions of the robot or 
the details of the communication, the motion language 
« accumulator includes a motion pattern used for directing 
the provision of a delimiter between the motion languag- 
es. By executing such a motion pattern for a delimiter 
between motion languages, the observer of the motion 
language can recognize the motion language without 
20 becoming confused. 

[0034] In still another form of the first aspect of the 
present invention, the articulated robot further includes 
a movement teaching section for directly teaching the 
motion pattern which is executed to realize the motion 
& language as a result of directly holding and operatinq 
each of the joints. 

[0035] In still another form of the first aspect of the 
present invention, the articulated robot further includes 
a movement teaching section for indirectly teaching the 
30 motion pattern which is executed to realize the motion 
language as a result of actuating each of the joints along 
a coordinate system of the robot. 
[0036] According to a second aspect of the present 
invention, there is provided method of controlling the 
35 motion of an articulated robot which comprises a plural- 
ity of joints and which accumulates a motion language 
which is formed by a motion pattern of each of the joints 
used to express the intentions of the robot and the de- 
tails of communication. The method comprises the 
*o steps of determining the motion language to be used, 
and controlling the movement of each of the joints for 
realizing the determined motion language. 
[0037] In one form of the second aspect of the present 
invention, the method further comprises the steps of in- 
45 putting and outputting data to the outside through a com- 
munications medium and of controlling communication 
based on the data input/output operation. In the method 
the motion language to be used is determined in accord- 
ance with the details of the communication. In addition 
so the inputting and outputting of data includes inputting of 
an image, inputting/outputting of sound, and transmit- 
ting/receiving of data. 

[0038] In still another form of the second aspect of the 
present invention, the method further comprises the 
55 step of executing a motion pattern used to direct the 
starting of the realization of the motion language and/or 
the step of executing the motion pattern used to direct 
the ending of the realization of the motion language. For 
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example, the starting and the ending of the realization 
of the motion language can be indicated by the raising 
of the right hand and the left hand of the humanoid robot. 
By executing the motion patterns thereof in starting and 
ending the communication, the observer of the motion 
language can easily and reliably recognize the motion 
language. 

[0039] In still another form of the second aspect of the 
present invention, the method comprises the step of, 
when the intentions of the robot or details of the com- 
munication are expressed by combining a plurality of the 
motion languages, executing a motion pattern used to 
direct the provision of a delimiter between the motion 
languages. By inserting a motion pattern for a delimiter 
between motion languages, the observer of the motion 
languages can recognize the motion pattern without be- 
coming confused. 

[0040] In still another form of the second aspect of the 
present invention, the method further comprises the 
step of directly teaching the motion pattern which is ex- 
ecuted to realize the motion language as a result of di- 
rectly holding and operating each of the joints. 
[0041] In still another form of the second aspect of the 
present invention, the method comprises the step of in- 
directly teaching the motion pattern which is executed 
to realize the motion language as a result of actuating 
each of the joints along a coordinate system of the robot. 
[0042] The present invention is widely applicable to 
an articulated robot comprising a plurality of joints. More 
preferably, it is applicable to a legged mobile robot such 
as a humanoid robot which has relatively long arms and 
legs and which can execute a motion pattern which is 
highly expressive of emotions and feelings. 
[0043] A humanoid robot can realize a motion lan- 
guage by executing a motion pattern using the four limbs 
and the trunk or a time-series change in a joint angle. 
Here, "motion language" refers to a pseudolanguage 
used to express the intentions of the robot and the de- 
tails of the communication to an external device or a us- 
er by using a predetermined motion pattern of the four 
limbs and the trunk. The motion language makes it pos- 
sible to transmit information without using any digital da- 
ta such as a program code and a communications code. 
The motion language is based on, for example, a meth- 
od in which the basic formation mode is in character 
units, and the robot 1 00 repeatedly forms a plurality of. 
characters in order to form a sentence, or a method in 
which communication is realized by a motion which has 
a meaning or based on a time-series change in a joint 
angle. 

[0044] By using the motion language which is made 
up of, for example, a motion pattern which is an approx- 
imation of the contour/form of a character, it can be vis- 
ually recognized and interpreted. By visually recogniz- 
ing and interpreting the contour/form which is represent- 
ed by the motion pattern, the meaning and character 
represented by each motion pattern can be determined 
even by a human being or a robot which does not pos- 



sess the same motion language database. 
[0045] For example, when the robot steps into a dan- 
gerous working area, it can give a message, such as the 
condition of the working area, to an observer at a remote 
s place without using any data communications means. 
[0046] The invention will now be described by way of 
example with reference to the accompanying drawings, 
throughout which like parts are referred to by like refer- 
ences, and in which: 
10 [0047] Fig. 1 is a front view of a legged mobile robot 
100 of an embodiment of the present invention. 
[0048] Fig. 2 is a back view of the legged mobile robot 
100 of the embodiment of the present invention. 
[0049] Fig. 3 is a schematic view of a degree-of-f ree- 
fs dom structural modal which the legged mobile robot 1 00 
of the embodiment uses. 

[0050] Fig. 4 is a schematic view of the structure of a 
system for controlling the legged mobile robot 1 00 of the 
embodiment. 

20 [0051 ] Fig. 5 is a schematic view of the flow of control 
data between a thinking controlling module 200 and a 
movement controlling module 300 inside the legged mo- 
bile robot 100 of the embodiment. 
[0052] Fig. 6 shows a flowchart illustrating the proce- 

25 dures for teaching a motion language to the legged mo- 
bile robot 100. 

[0053] Fig. 7 illustrates the motion patterns of the leg- 
ged mobile robot 1 00 for forming letters (from A to N) of 
the alphabet, in a method used to realize a motion lan- 

30 guage one character at a time. 

[0054] Fig. 8 illustrates the motion patterns of the leg- 
ged mobile robot 100 for forming letters (from 0 to Z) of 
the alphabet, in the method used to realize a motion lan- 
guage one character at a time. 

35 [0055] Fig. 9 illustrates an example of transforming 
the English word "HELLO" to a motion language in ac- 
cordance with the method used to realize a motion lan- 
guage one character at a time. 
[0056] Fig. 1 0 illustrates in the form of transitions from 

40 one state to another the procedures for forming the Eng- 
lish word "HELLO" which is formed in accordance with 
the method used to realize a motion language one char- 
acter at a time. 

[0057] Fig. 1 1 illustrates in the form of transitions from 
45 state to another the procedures for recognizing the Eng- 
lish word "HELLO" which is formed in accordance with 
the method used to realize a motion language one char- 
acter at a time. 

[0058] Fig. 12 illustrates an example of transforming 
so a communication message "Come immediately" to mo- 
tion languages in accordance with a method used to re- 
alize communication by a motion which has a meaning 
or a time-series change in a joint angle. 
[0059] Fig. 13 illustrates in the form of transitions from 
55 one state to another the procedures for expressing the 
communication message "Come immediately" which is 
expressed in accordance with the method used to real- 
ize communication by a motion which has a meaning or 
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a time-series change in a joint angle. 
[0060] • Fig. 14 illustrates in the form of transitions from 
one state to another the procedures for recognizing the 
communication message "Come immediately" which is 
expressed in accordance with the method used to real- 
ize communication by a motion which has a meaning or 
a time-series change in a joint angle. 
[0061] Hereunder, a description of a preferred em- 
bodiment of the present invention will be given in detail 
with reference to the drawings. 
[0062] Figs. 1 and 2 are front and back views showing 
a humanoid legged mobile robot 1 00 of an embodiment 
of the present invention which is in an upright posture. 
As shown in the figures, the legged mobile robot 1 00 
comprises the two lower limbs or left and right.legs used 
for movement, a trunk, left and right upper limbs, and a 
head. 

[0063] The left and right lower limbs each comprise a 
thigh, a knee joint, a shank, an ankle, and a foot. They 
are connected to substantially the bottom end of the 
trunk by corresponding hip joints. The left and right up- 
per limbs comprise corresponding upper arms, elbow 
joints, and forearms. They are connected to their corre- 
sponding left and right upper side edges of the trunk by 
corresponding shoulder joints. The head is connected 
to substantially the uppermost end center portion of the 
trunk by a neck joint. 

[0064] A control system which cannot be seen in Figs. 
1 and 2 is disposed inside a trunk unit. The control sys- 
tem is a case in which a controller (a main control sec- 
tion) for controlling the actuation of each joint actuator 
making up the legged mobile robot 1 00 and for process- 
ing externally input information from, for example, each 
sensor (described later), and a peripheral device such 
as a power supply circuit are installed. The control sec- 
tion may also include a remote-control communications 
interface or a communications device. 
[0065] Fig. 3 schematically illustrates a joint structure 
of the legged mobile robot 100, which provides the de- 
grees of freedom thereof. As shown in Fig. 3, the legged 
mobile robot 100 comprises an upper part of the body 
including two arms and a head 1 , lower limbs or two legs 
used for movement, and a trunk which connects upper 
limbs and the lower limbs. 

[0066] A neck joint which supports the head 1 pos- 
sesses three degrees of freedom which are provided in 
correspondence with a neck joint yaw axis 2, a neck joint 
pitch axis 3, and a neck joint roll axis 4. 
[0067] Each arm includes a shoulder joint pitch axis 
8, a shoulder joint roll axis 9, an upper arm yaw axis 1 0, • 
an elbow joint pitch axis 11, a forearm yaw axis 12, a 
wrist joint pitch axis 13, a wrist joint roll axis 14, and a 
hand 15. Each hand 15 is actually a structure which has 
many joints and degrees of freedom including a plurality 
of fingers. However, since the motion of each hand 15 i 
itself rarely contributes to and influences the stable pos- 
ture control operations and the walking controlling op- 
erations of the robot 100, each hand in the embodiment 



is assumed to possess zero degrees of freedom. There- 
fore, in the embodiment, each of the left and right arms 
possesses seven degrees of freedom. 
[0068] The trunk possesses three degrees of freedom 
s which are provided in correspondence with a trunk pitch 
axis 5, a trunk roll axis 6, and a trunk yaw axis 7. 
[0069] The left and right legs comprising the lower 
limbs each include a hip joint yaw axis 16, a hip joint 
pitch axis 17, a hip joint roll axis 18, a knee joint pitch 
10 axis 1 9, an ankle joint pitch axis 20, a joint roll axis 21 , 
and a foot (or a sole) 22. The points where the hip joint 
pitch axes 1 7 and their corresponding hip joint roll axes 
1 8 intersect are defined as the locations of the hip joints 
of the robot 1 00 in the embodiment. The feet (or soles) 
'5 22 ofthe human body are actually structures which have 
many joints and degrees of freedom including the soles. 
However, the soles of the legged mobile robot 100 of 
the embodiment are assumed as having zero degrees 
of freedom. Therefore, in the embodiment, each of the 
v left and right legs possesses six degrees of freedom. 
[0070] Tosum up, the total number of degrees of free- 
dom of the whoie legged mobile robot 100 of the em- 
bodiment is 3 + 7x2 + 3 + 6x2 = 32. However, the 
number of degrees of freedom of the legged mobile ro- 
5 bot 100 is not necessarily limited to 32. It is obvious that 
the number of degrees of freedom, that is, the number 
of joints can be increased or decreased as necessary 
in accordance with, for example, the specification re- 
quirements and the limiting conditions in designing and 
30 manufacturing the robot. 

[0071] Each degree of freedom of the above-de- 
scribed legged mobile robot 100 provided by each joint 
is actually provided by the active operation of its corre- 
sponding actuator. To respond to various demands such 
35 as of approximating the form of the robot to the natural 
form of a human being by removing extra bulges from 
the external appearance of the device, and of controlling 
the posture of an unstable structure which walks using 
two legs, it is preferable to use small and light joint ac- 
40 tuators. In the embodiment, there are used in the robot 
100 small AC servo actuators which are directly con- 
nected to gears and which incorporate in a motor unit a 
servo control system formed into a one-chip system. A 
small AC servo actuator which is applicable to a legged 
'5 robot is disclosed in, for example, Japanese Patent Ap- 
plication No. 11-33386 which has already been as- 
signed to the applicant. 

[0072] Fig. 4 is a schematic view of the structure of a 
controlling system of the legged mobile robot 1 00 of the 

«> embodiment of the present invention. In the figure, the 
system comprises a thinking controlling module 200 
which directs the expression and determination of emo- 
tions and feelings by dynamically reacting to, for exam- 
ple, the input operation performed by the user, and a 

5 movement controlling module 300 for controlling the 
harmonious movement of the whole body of the robot 
by, for example, actuating a joint actuator. 
[0073] The thinking controlling module 200 is an inde- 
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pendent information processing device which can per- 
form a selNcompleting operation and which comprises 
a CPU (central processing unit) 211 for executing a com- 
puting operation regarding the expression and determi- 
nation of feelings and emotions, RAM (random access 
memory) 212, ROM (read only memory) 213, and an 
external storage device (such as a hard disk drive) 214. 
Motion language database (described later) for realizing 
communication by a motion can be accumulated in the 
external storage device 214. 

[0074] Through a bus interface 201 , various devices 
such as a communications interface 254 which ex- 
changes data with systems outside the robot 1 00 are 
connected to the thinking controlling module 200. The 
communications interface 254 performs the exchange 
of data through, for example, an image input device 251 
such as a CCD (charge coupled device) camera, a 
sound input device 252 such as a microphone, a sound 
outputting device 253 such as a speaker, and LAN (local 
area network) which is not shown. 
[0075] At the thinking controlling module 200, the 
present feelings and intentions of the legged mobile ro- 
bot 1 00 are determined in accordance with, for example, 
external stimulus, such as the input of visual data from 
the image input device 251 or the input of hearing data 
from the sound input device 252. In addition, a com- 
mand is issued to the movement controlling module 300 
so that the robot acts, that is, moves using the four limbs, 
based on the decision made. 

[0076] On the other hand, the movement controlling 
module 300 is a separate information processor for per- 
forming a self -completing operation. It comprises a CPU 
(central processing unit) 311 for controlling the harmo- 
nious movement of the whole body of the robot 100, 
RAM (random access memory) 312, ROM (read only 
memory) 313, and an external storage device (such as 
a hard disk drive) 31 4. For example, time-series chang- 
es in a joint angle or motion patterns used for realizing 
the motion language using the four limbs can be accu- 
mulated in the external storage device 314. The motion 
language is a pseudolanguage used to express the in- 
tentions of the robot and the details of the communica- 
tion to an external device or a user by using a predeter- 
mined motion pattern of the four limbs and the trunk. 
Specific examples of the various motion languages will 
be given later. 

[0077] Through a bus interface 301, the movement 
controlling module 300 has connected thereto various 
devices such as joint actuators for providing degrees of 
freedom at the joints (see Fig. 3) which are spread 
throughout the whole body of the robot 100, a posture 
sensor 351 for measuring the posture and inclination of 
the trunk, floor contact confirmation sensors 352 and 
353 for detecting the contacting of and moving off of the 
soles of the left foot and right foot with and from the floor, 
and a power supply controller for controlling the power 
supply such as a battery. 

[0078] At the movement controlling module 300, the 



harmonious movement of the whole body by each of the 
joint actuators is controlled while realizing the action 
prescribed by the thinking controlling module 200. In 
other words, the CPU 311 takes out a motion pattern in 

5 accordance with the action prescribed by the thinking 
controlling module 200 from the external storage device 
314 or internally generates a motion pattern. In accord- 
ance with the specified motion pattern, the CPU 311 
sets, for example, the movement of the feet, the path of 

10 the ZMP (zero moment point), the movement of the 
trunk, the movement of the upper limbs, the horizontal 
position and height of the waist, etc., and sends to each 
of the joint actuators a command value for instructing 
the execution of the pattern in accordance with the de- 

15 tails of these settings. (ZMP in the specification is the 
point on the floor surface where the moment resulting 
from the floor reaction force when the robot walks is ze- 
ro. The ZMP path refers to the path of movement of the 
ZMP when, for example, the robot 1 00 is walking.) 

20 [0079] By detecting the tilting and posture of the trunk 
of the robot 1 00 by the output signal of the posture sen- 
sor 351 and by detecting whether each movable leg is 
either in a "swinging state" or a "standing state" by an 
output signal of each of the floor contact confirmation 

25 sensors 352 and 353, the CPU 311 can appropriately 
control the harmonious movement of the whole body of 
the legged mobile robot 1 00. 

[0080] The movement controlling module 300 returns 
to the thinking controlling module 200 information con- 

30 ceming to what degree the robot has acted in accord- 
ance with the decision made at the thinking controlling 
module 200, that is, in accordance with the information 
concerning the processing state. 
[0081] The thinking controlling module 200 and the 

35 movement controlling module 300 are constructed on a 
common platform, and are connected together through 
the bus interfaces 201 and 301 . 
[0082] In the case where, as shown in Fig. 3, the leg- 
ged mobile robot 1 00 has a bending portion (that is, the 

40 trunk pitch axis 5 and/or the trunk roll axis 6) at the trunk 
thereof, when the thinking controlling module 200 is dis- 
posed above the bending portion, and the movement 
controlling module 300 is disposed below the bending 
portion, a wiring which passes by the bending portion 

45 only serves as a bus for linking both of these modules. 
Therefore, the number of wires is decreased, thereby 
simplifying the electrical component/wiring design near 
the bending portion. 

[0083] As can be seen from Fig. 4, the thinking con- 
50 trolling module 200 comprises many wires which are 
used to make connections with various devices which 
are disposed in the head of the robot 100, such as an 
image input device 251 and a sound input device 252 
which correspond to the sense of sight and the sense 
55 of hearing of a human being, respectively, and a sound 
outputting device 253. Since the main movement sys- 
tem of the legged mobile robot 100 is the movable legs, 
the movement controlling module 300 includes a very 
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large number of wires which extend towards the lower 
half of the body of the robot 1 00. Therefore, by disposing 
the thinking controlling module 200 above the trunk, and 
the movement controlling module 300 below the trunk, 
the total lengths of the wires of the entire system can be 
decreased, thereby making it possible to simplify the 
wiring structure. 

[0084] Fig. 5 schematically illustrates the flow of con- 
trol data between the thinking controlling module 200 
and the movement controlling module 300 inside the 
legged mobile robot 100. 

[0085] At the movement controlling module 300, the 
harmonious movement of the whole body is controlled 
by actuating each of the joint actuators in synchronism 
with each other while realizing the action prescribed by 
the thinking controlling module 200 through a movement 
output controller 234. More specifically, in accordance 
with the specified motion pattern, the movement of the 
legs, the ZMP (zero moment point) path, the movement 
of the trunk, the movement of the upper limbs, the hor- 
izontal position and height of the waist, etc., are set, and 
a command value used to instructthe execution of amo- 
tion in accordance with the details of the settings is sent 
to each of the joint actuators. 
[0086] From an encoder which is provided at each 
joint actuator, the rotational angle data of each joint is 
returned. As a result, the movement controlling module 
300 can know the actual displacement amount and dis- 
placement speed of each joint. In addition, by subjecting 
geometrical data concerning, for example, each joint 
displacement angle and each arm length to a predeter- 
mined geometrical computation, the pattern of the actu- 
al movement of the whole body of the legged mobile ro- 
bot 100 can be obtained. The encoder output of each 
joint actuator is sent towards the thinking controlling 
module 200 in order to teach the movement. 
[0087] Detection signals from the posture sensor 351 
and the floor contact confirmation sensors 352 and 353 
for detecting the contacting of the left and right soles 
with the floor are input to the movement controlling mod- 
ule 300. Based on these detection signals and the actual 
motion pattern which has been calculated, the harmo- 
nious movement of the whole body of the legged mobile 
robot 1 00 can be appropriately controlled. 
[0088] Although not shown in Fig. 4, a power sensor, 
a contact sensor, a temperature sensor, etc., may be 
further disposed at predetermined portions of the legged 
mobile robot 100. The movement controlling module 
300 can use sensation-of-power data, tactile data, and 
temperature data taken in from these sensors in order 
to appropriately perform a controlling operation. 
[0089] Based on external stimulus such as sound or 
something affecting the sense of vision (for example, 
stimulus obtained from a nearby user), the thinking con- 
trolling module 200 can determine the present feelings 
and intentions of the legged mobile robot 100. In other 
words, the thinking controlling module 200 takes in im- 
age data as visual data which is acquired at the image 



input device 251 , and sound data as hearing data which 
Is input to the sound input device 252. In accordance 
with such external stimuli, the present feelings and in- 
tentions of the legged mobile robot 1 00 are determined. 
5 [0090] The thinking controlling module 200 can also 
determine the present feelings and intentions of the leg- 
ged mobile robot 1 00 in accordance with data or a com- 
mand transmitted from an external system through the 
communications interface 254, without depending upon 
io external stimuli which are directly input to the legged 
mobile robot 100. In this case, a user at a remote place 
can supply data regarding, for example, the five senses 
of the robot 1 00 or transmit a command to make it pos- 
sible to prompt the robot 100 to make a decision. 
'5 [0091 ] Further, the thinking controlling module 200 is- 
sues a command to the movement controlling module 
300 so that the action or conduct, that is, movement us- 
ing the four limbs based on the decision made is exe- 
cuted. 

20 [0092] Still further, the thinking controlling module 200 
not only prompts a movement controlling operation in 
accordance with the decision made by the module 200 
itself, but also can cause sound data which expresses 
the intentions and the feelings of the robot to be output 
25 from the sound outputting device 253. 

[0093] Still further, the thinking controlling module 200 
has the ability to communicate by a motion, and, thus, 
can direct the movement controlling module 300 to ex- 
ecute a language realization motion pattern. In order to 
30 realize communication by a motion, the thinking control- 
ling module 200 comprises a command analyzer 231 , a 
communication controller 232, a motion language data- 
base 233, a movement output controller 234, and a 
movement teaching section 235. 
35 [0094] The command analyzer 231 interprets a com- 
mand input in an electronic data format thro ugh the com- 
munications interface 254, or a command input as a 
sound command through the sound input device 252. 
The command analyzer 231 can also interpret a com- 
40 mand written in a visual information format such as a 
cyber code as a result of inputting the command to the 
image input device 251. 

[0095] When, as a result of interpreting the command, 
the command analyzer 231 determines that the com- 
45 mand is a motion command in which a particular result 
is directly expected, it gives the result of the interpreta- 
tion to the movement output controller 234. In contrast, 
when the command analyzer 231 determines that the 
command is a command used for communication by a 
50 motion (a motion language command) instead of a com- 
mand in which a particular result is directly desired, it 
gives the result of the interpretation to the communica- 
tion controller 232. 

[0096] When sound is used for communication, the 
55 communication controller 232 outputs the details of the 
communication using sounds through the sound output- 
ting device 253. On the other hand, when communica- 
tion is carried out using a language which is realized 
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based on a motion pattern, the communication control- 
ler 232 searches the motion language database 233 in 
order to take out the appropriate motion pattern or a 
time-series change in a joint angle, that is, a motion lan- 
guage, and gives this to the movement output controller 
234. 

[0097] The communication controller 232 may control 
bidirectional communication between the robot 1 00 and 
an external device or a user. In this case, it is preferable 
not only to include a mechanism for determining the mo- 
tion command issued to the movement controlling mod- 
ule 300, but also to include, for example, a mechanism 
for managing the history of the motion commands which 
have been issued in the past, and a mechanism for stud- 
ying the determination of the motion commands based 
on these command histories. 

[0098] The movement output controller 234 transmits 
the motion command or motion language determined at 
the thinking controlling module 200 to the movement 
controlling module 300 at a predetermined suitable tim- 
ing. 

[0099] The motion language which accumulates in 
the motion language database 233 is a pseudolanguage 
which is realized by amotion pattern using the four limbs 
of the legged mobile robot 1 00 or based on a time-series 
change in a joint angle. The motion language allows in- 
formation to be transmitted without using program 
codes, communication codes, or any other such types 
of digital data. The details of the motion language are 
discussed later. 

[01 00] Such motion language database can be accu- 
mulated in, for example, an external storage device 21 4 
(or a ROM 213) in a nonvolatile manner. At the time of 
shipment, the motion language database can be 
preloaded in the external storage device 21 4. Aftership- 
ment, the motion language database can be installed in 
the external storage device 214 through a portable re- 
cording medium such as a CD (compact disc), an MO 
(magneto-optical disc), and a DVD (digital versatile 
disc). In addition, a motion language database can be 
downloaded from a server (not shown) of a network 
through the communications interface 254. 
[0101] Further, the motion language can be taught to 
the actual machine, that is, the legged mobile robot 1 00. 
One example of a teaching method which may be used 
. involves the user (operator) directly holding and manu- 
ally operating each of the joints of, for example, the 
hands and feet of the robot 1 00 in order to directly teach 
the motion pattern used for realizing the motion lan- 
guage. Another example of a teaching method which 
may also be used involves the actualization of each of 
the joints of, for example, the hands and feet of the robot 
100 in accordance with the orthogonal coordinate sys- 
tem through a user Input device, such as a commander, 
in order to indirectly teach the motion pattern used to 
realize the motion language. 

[0102] Fig. 6 illustrates in flowchart form the proce- 
dures for teaching the motion language to the legged 



mobile robot 100. Using any teaching method, such as 
a teaching playback method or an online method, the 
operator or user of the robot 100 can input the motion 
language with respect to the four limbs of the legged 

5 mobile robot 1 00 (Step S1 ). During the inputting opera- 
tion, an angle of rotation which defines the motion pat- 
tern or the time-series change in a joint angle is output 
from the encoder of each joint actuator (Step S2). The 
operator provides a meaning to the realization of the lan- 

w guage through the movement teaching section 235 with 
respect to the taught motion pattern or the time-series 
change in a joint angle. As a result of this, the rotational 
angle data which defines the motion pattern or the time- 
series change in a joint angle is made to correspond to 

15 the realization of the language, and is recorded in the 
database as a motion language (Step S3). 
[0103] Motion language in the present Invention re- 
fers to a pseudolanguage for expressing to, for example, 
an external device or a user the intentions or the details 

20 of the communication using a predetermined motion 
pattern using the four limbs and the trunk. The motion 
language makes it possible to transmit information with- 
out using any digital data such as a program code and 
a communications code. The motion language in the 

25 embodiment is based on, for example, a method in 
which the basic realization mode is in character units, 
and the robot 100 repeatedly forms a plurality of char- 
acters in order to form a sentence, or a method in which 
communication is realized by a motion which has a 

30 meaning or based on a time-series change in a joint an- 
gle. 

[0104] . In the former case where the motion language 
is based on a method in which the basic realization 
mode is in character units, characters are combined in 

35 order to make it possible to form an unlimited number 
of words and sentences. In other words, the motion lan- 
guage makes it possible to achieve an infinite number 
of types of communication. However, in order to make 
the robot 1 00 communicate once, it is necessary to ex- 

40 ecute a motion pattern for every character of a sentence 
which expresses the details of the communication, so 
that it takes a relatively long time to realize the motion 
language. 

[01 05] Figs. 7 and 8 illustrate examples of motion pat- 
45 terns of the legged mobile robot 1 00 for forming the let- 
ters of the alphabet using the method in which the basic 
realization mode of the motion language is in character 
units. It is to be understood that, by successively exe- 
cuting the motion pattern which corresponds to each let- 
so ter of the alphabet, any type of words and sentences 
can be formed by the robot 100. 
[0106] In the examples shown in Figs. 7 and 8, each 
of the letters is formed by causing the legged mobile ro- 
bot 1 00 to form each motion pattern which is an approx- 
55 Imation of the shape/contour of each letter, using the 
four limbs and the trunk. Therefore, even if robots which 
do not have the same motion language databases are 
used, or even if it is a human being who cannot possess 
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a motion language database in the first place, the char- 
acters which are indicated by the corresponding motion 
patterns can be determined by visually recognizing and 
interpreting the contours/shapes which the correspond- 
ing motion patterns indicate. For example, a robot which 
has stepped into a dangerous working area can visually 
convey in character units to a human being which ob- 
serves or watches from a remote place a message re- 
garding, for example, the condition of the working place 
without using any data communications means such as 
wired or radio data communication means (or without 
establishing any communications connection). 
[0107] However, in the present invention, the motion 
patterns used to form the letters are not limited to those 
illustrated above. The letters of the alphabet can be 
formed using motion patterns other than those illustrat- 
ed in Figs. 7 and 8. Although the motion patterns used 
to form the letters of the alphabet are only illustrated, it 
is to be sufficiently understood that the motion patterns 
of the characters of language systems other than the 
letters of the alphabet, such as arabic numerals, Hira- 
gana characters, Katakana characters, Kanji charac- 
ters, and Hankul characters can also be formed in order 
to realize motion languages by causing the legged mo- 
bile robot 100 to realize the motion patterns which are 
approximations of the contours/shapes of the charac- 
ters (or simplified forms of the characters), using the four 
limbs and the trunk. 

[0108] In the latter case where the motion language 
is based on a method in which communication is real- 
ized by a motion which has a meaning or based on a 
time-series change in a joint angle, it is not necessary 
to form motion patterns which are imitations of the forms 
and contours of characters one character at a time, so 
that the motion language can be realized in a relatively 
short time. However, the number of types of communi- 
cation that can be realized is limited to the number of 
types of communication previously provided in the mo- 
tion language database 233. 
[0109] Actually, a hybrid method which is a combina- 
tion of the method used to realize the motion language 
one character at a time and the method used to realize 
communication by a motion which has a meaning or 
based on a time-series change in a joint angle can be 
used in order to realize the motion language in a rela- 
tively short time and to realize many motion languages 
which are not provided in the motion ' language data- 
base 233. In other words, the hybrid method allows a 
motion language to be realized highly efficiently for more 
general purposes. 

[01 10] Fig. 9 illustrates an example of transforming an 
English word "HELLO" into a motion language in ac- 
cordance with the method of realizing the motion lan- 
guage one character at a time. 
[01 1 1 ] When the legged mobile robot 1 00 is executing 
any work or movement, the communication controller 
232 starts executing the operation for realizing a motion 
language in response to the determination of forming 



the English word "HELLO." 

[0112] First, the motion pattern of raising the right 
hand is executed in order to indicate the start of the re- 
alization of the motion language to an external device 
5 or the like. 

[0113] Then, after the robot 100 has assumed a ver- 
tical posture which signifies a delimiter between char- 
acters the first letter "H" is formed as a result of a motion. 
The rotational position of each joint actuator which de- 
10 fines the motion pattern corresponding to that of the let- 
ter "H" and the time-series change in a joint angle for 
proceeding from the vertical posture to this motion pat- 
tern'are accumulated in the motion language database 
233. They are transmitted to the movement controlling 
is module 300 through the movement output controller 
234. 

[0114] Thereafter, the robot 100 successively forms 
as a result of motions the letters "E", "L", V, and "O", 
with the robot 1 00 assuming a vertical posture which sig- 
20 nifies a delimiter between letters. 

[0115] The rotational position of each joint actuator 
which defines the motion pattern which corresponds to 
that of each of the letters "E", V, and "E" and the time- 
series change in a joint angle for proceeding from the 
25 vertical posture to the corresponding motion pattern are 
accumulated in the motion language database 233. 
They are successively transmitted to the movement 
controlling module 300 through the movement output 
controller 234. 

30 [0116] Lastly, the motion pattern of raising the left arm 
is executed in order to indicate to an external device or 
the like that the operations for realizing the motion lan- 
guage are completed. 

[01 1 7] Although in the example of Fig. 9 the robot 1 00 
35 is made to assume a vertical posture which signifies a 
delimiter between letters, whether or not to insert a de- 
limiter between letters is optional. However, when the 
robot 100 is made to assume a delimiter posture, the 
observer of the motion language can clearly visually rec- 
40 ognize the delimiter between characters. When the 
same letter, such as the letter V in Fig. 9, appears suc- 
cessively two or more times in a row, the insertion of a 
delimiter makes it possible to avoid confusion. It is nec- 
essary to specially determine the required time for form- 
45 ing a character by the observer and the side which re- 
alizes the motion language. By using the motion pattern • 
of proceeding from the vertical posture to the posture 
which forms a character, and the motion pattern of pro- 
ceeding from the posture which forms the character to 
so the vertical posture, any combination of characters can 
be formed, thereby making it possible to decrease the 
total number of pieces of motion pattern data which are 
used. 

[0118] It is not always necessary to express the de- 
« limiter between characters (or motion languages) by a 
vertical posture, so that other postures and motion pat- 
terns may be used to express the delimiter. 
[01 1 9] Fig. 1 0 illustrates in the form of transitions from 



10 



19 EP11 

one state to another the procedures for forming the Eng- 
lish word "HELLO" which is formed in accordance with 
the method of realizing the motion language one char- 
acter at a time. 

[0120] In State 0 in which another motion basic pos- 
ture is assumed, the command analyzer 231 analyzes 
an external command, and issues a command for form- 
ing the English word "HELLO" using a motion language . 
which is realized one character at a time to the commu- 
nication controller 232. Each letter of the English word 
is buffered. 

[0121] When a motion realization command is issued, 
the robot temporarily stops the task now being execut- 
ed. The state changes to State 1 in order to reproduce 
a motion language realization start motion. In the exam- 
ple shown in Fig. 9, the motion language realization start 
motion corresponds to the raising of the right hand. By 
reading out the proper motion data from the motion lan- 
guage database 233, and transmitting this to the move- 
ment controlling module 300, the motion language real- 
ization start motion is reproduced. 
[0122] When the reproduction of the motion language 
realization start motion is completed, the state changes 
to State 2 in order to realize a movement basic posture 
during the realization of the motion language. The 
movement basic posture corresponds to the vertical 
posture in the example shown in Fig. 9, and is a delimiter 
between letters (or motion languages). 
[0123] At this time, a confirmation is made as to 
whether the content of the buffer which has received a 
formation command corresponds to that of the word 
"HELLO." Then, the first letter TP is taken out therefrom 
in order for the state to change to State 3. 
[0124] In State 3, the motion data which corresponds 
to that of the letter "H" is read out from the motion lan- 
guage database 233, and is transmitted to the move- 
ment controlling module 300. Then, the state changes 
to State 4 in order to reproduce this motion language 
realization motion. 

[0125] When the reproduction of this motion language 
realization motion is completed, the state is restored to 
State 2. By executing the movement basic posture dur- 
ing the realization of the motion language, the delimiter 
between characters (or motion languages) is ex- 
pressed. Thereafter, a confirmation is made as to wheth- 
er the content of the buffer which stores the formation 
command corresponds to the word "HELLO" in order to 
take out the next letter, after which the state changes to 
State 3 in order to reproduce the next motion language 
realization motion. 

[0126] The reproduction of the motion language real- 
ization motion is repeated until all of the letters (or the 
motion language) in the buffer which stores the forma- 
tion commands have been formed. Therefore, in the ex- 
ample shown in Fig. 9, the letters "H", "E", V, "L", and 
"0" are successively formed in that order. 
[0127] When all of the letters (or the motion language) 
have been read out from the buffer which stores the for- 
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mation commands, and the reproduction of the motion 
language realization motions of these letters have been 
completed, the state changes to State 5 via the move- 
ment basic posture (that is, the vertical posture) in State 
5 2. 

[0128] In State 5, a motion language realization end 
motion which indicates the end of the realization of the 
motion language is reproduced. In the example shown 
in Fig. 9, the motion language realization end motion 

10 corresponds to the raising of the left hand. By reading 
out the proper motion data from the motion language 
database 233, and transmitting it to the movement con- 
trolling module 300, the motion language realization end 
motion is reproduced. 

is [0129] Then, the state is restored to State 0 in order 
to reproduce the movement basic posture just before 
the Issuance of the motion language realization com- 
mand. In other words, the robot can restart the execution 
of the task which was temporarily stopped. 

20 [0130] For example, an external device (such as an- 
other robot) having a similar motion language database 
can visually recognize the motion language illustrated 
in Fig. 9. Fig. 11 illustrates in the form of transitions from 
one state to another the procedures for recognizing the 

25 English word "HELLO" which is formed in accordance 
with the method used to realize the motion language one 
character at a time. 

[01 31 ] In State 1 0 prior to the recognition of the motion 
language, the movement of the legged mobile robot 1 00 

30 which realizes the motion language is visually observed 
using, for example, a camera. When the motion is rec- 
ognized, the state changes to State 11 in order to rec- 
ognize the motion language realization start motion. 
When the start motion (which corresponds to the raising 

35 of the right hand in the example shown in Fig. 9) stored 
in the motion language database 233, and an image in- 
put through the camera are compared and do not match, 
the state returns to State 10 to execute a waiting oper- 
ation. 

40 [0132] On the other hand, when the motion language 
realization start motion and the image are compared 
and match, the state changes to State 12 in response 
to the completion of the reproduction of the motion lan- 
guage realization start motion. In State 12, when a mo- 

45 tion other than the motion during the recognition of the 
motion language is recognized, and this motion differs 
from the motion language end motion (which corre- 
sponds to the raising of the left hand in the example 
shown in Fig. 9), the state changes to State 13. 

50 [0133] In State 13, the motion language database is 
searched in order to determine the motion language (in 
this case, one character) which corresponds to that of 
the recognized motion. 

[0134] The recognized character is temporarily 
55 stored. The state returns to State 12 in order to re-exe- 
cute the operation for determining the motion language 
after waiting for the motion recognition operation. The 
motion recognition operation and the operation for de- 
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termining the motion language are repeated until the ro- 
bot which realizes the motion language reproduces the 
motion language realization end motion. 
[0135] Therefore, in the example illustrated in Fig 9 
the letters "H", »E", "L", V, and "0" are successively de- 
tenmined in that order. Newly determined characters are 
successively added to the end of the character string. 
As a result, the character string making up the commu- 
nication message, that is, the word "HELLO" is ob- 
tained. 

[0136] In State 12, when a motion other than the mo- 
tion during the recognition of themotion language is rec- 
ognized, and a determination is made that the motion 
corresponds to the motion language end motion (which 
corresponds to the raising of the left hand in the example 
shown in Fig. 9), the state changes to State 14 in order 
to set the recorded character string (that is, the word 
"HELLO" in the example shown in Fig. 9) as a return 
value, after which the state is restored to State 10. 
[0137] When the side which recognizes the motion 
language is of the same type as the legged mobile robot 
1 00, the communication controller 232 may make a de- 
cision in response to the obtainment of a term which has 
a meaning as a return value. In accordance with the de- 
cision made, the movement controlling module 300 may 
be directed to execute the motion pattern (in this case, 
greeting someone, etc.) in response to a sound output 
or an input communication. 

[0138] Fig. 12 illustrates an example in which the 
communication message "Come immediately" is trans- 
formed into a motion language in accordance with the 
method used to realize communication by motions 
which have meanings or based on time-series changes 
in actuator angles. In the embodiment, the communica- 
tion message "Come immediately" is expressed as a 
combination of two types of motion languages, one of 
which means "come" and the other of which means "im- 
mediately." " 

[0139] First, the motion pattern of raising the right 
hand is executed in order to indicate the start of the re- 
alization of the motion languages. 
[0140] Next, after the robot 100 has assumed a verti- 
cal posture which signifies a delimiter between charac- 
ters, the motion language "immediately" is expressed. 
The rotational position of each joint actuator which de- 
fines the motion pattern corresponding to that of the mo- 
tion language "immediately" and the time-series change 
in a joint angle for proceeding from the vertical posture 
to this motion pattern are accumulated in the motion lan- 
guage database 233, and are transmitted to the move- 
ment controlling module 300 through the movement out- 
put controller 234. 

[0141] Then, after the robot 100 has assumed a ver- 
tical posture which signifies a delimiter between char- 
acters, the next motion language "come" is expressed. 
The rotational position of each joint actuator which de- 
fines the motion pattern corresponding to that of the mo- 
tion language "come" and the time-series change in a 



joint angle for proceeding from the vertical posture to 
this motion pattern are accumulated in the motion lan- 
guage database 233, and are transmitted to the move- 
ment controlling module 300 through the movement out- 
5 put controller 234. 

[0142] Lastly, the motion pattern of raising the left 
hand is executed in order to indicate to an external de- 
vice or the like the end of the realization of the motion 
language. 

"> [0143] Although in the example shown in Fig. 12 the 
robot 100 is made to assume a vertical posture which 
signifies a delimiter between characters, whether or not 
to insert a delimiter between motion languages is op- 
tional (as in the description above). It is not always nec- 
w essary to express the delimiter between motion lan- 
guages by a vertical posture, so that other postures may 
be used to express the delimiter. 
[0144] Fig. 13 illustrates in the form of transitions from 
one state to another the procedures for realizing the mo- 
20 tion languages "Come immediately" in accordance with 
the method used to realize communication by motions 
which have meanings or based on time-series changes 
in an actuator angle. 

[0145] In State 20 in which another basic posture is 
25 assumed, the command analyzer 231 analyzes an ex- 
ternal command. Then, the motion languages "Come 
immediately" which realize a communication by motions 
which have meanings or based on time-series changes 
in actuator angles are issued to the communication con- 
30 troller 232. Each of the motion languages "immediately" 
and "come" is buffered in that order. 
[0146] When a motion expression command is is- 
sued, the robot temporarily stops the task now being ex- 
ecuted. The state changes to State 21 to in order to re- 
35 produce a motion language realization start motion. In 
the example shown in Fig. 12, the motion language re- 
alization start motion corresponds to the raising of the 
right hand. By reading out the proper motion data from 
the motion language database 233, and transmitting this 
40 to the movement controlling module 300, the motion lan- 
guage realization start motion is reproduced. 
[0147] When the reproduction of the motion language 
realization start motion is completed, the state changes 
to State 22 in order to realize a movement basic posture 
45 during the realization of the motion language. The 
movement basic posture corresponds to the vertical 
posture in the example shown in Fig. 12, and is a delim- 
iter between motion languages. 
[0148] At this time, a confirmation is made as to 
so whether the content of the buffer which has received a 
realization command corresponds to the motion lan- 
guages "Come immediately." Then, the motion lan- 
guage "immediately" is taken out therefrom in order for 
the state to change to State 23. 
55 [0149] In State 23, the motion data which corresponds 
to that of the motion language "immediately" is read out 
from the motion language data base 233, and is trans- 
mitted to the movement controlling module 300. Then, 
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the state changes to State 24 in order to reproduce this 
motion language realization motion. 
[0150] When the reproduction of this motion language 
realization motion is completed, the state is restored to 
State22. By executing the movement basic posture dur- 5 
ing the realization of the motion language, the delimiter 
between motion languages is indicated. Thereafter, a 
confirmation is made on the content of the buffer which 
stores the realization command in order to take out the 
next motion language "come," after which the state 10 
changes to State 23 in order to feproduce the next mo- 
tion language realization motion. 
[0151] The reproduction of the motion language real- 
ization motion is repeated until all of the letters (or the 
motion languages) in the buffer which stores the reali- « 
zation commands have been processed. Therefore, in 
the example shown in Fig. 1 2, the motion languages "im- 
mediately" and "come" are successively formed in that 
order. 

[0152] When the motion languages have been read 
out from the buffer which stores the realization com- 
mands, and the reproduction of the motion language re- 
alization motions of these languages has been complet- 
ed, the state changes to State 25 via the movement ba- 
sic posture (that is, the vertical posture) in State 22. 
[0153] In State 25, a motion language realization end 
motion which indicates that the realization of the motion 
languages is completed is reproduced. In the example 
shown in Fig. 12, the motion language realization end 
motion corresponds to the raising of the left hand. By 
reading out the corresponding motion data from the mo- 
tion language database 233, and transmitting this to the 
movement controlling module 300, the motion language 
realization end motion is reproduced. Then, the state is 
restored to State 20 in order to reproduce the movement 
basic posture just before the issuance of the motion lan- 
guage realization command. In other words, the robot 
can restart the execution of the task which was tempo- 
rarily stopped. 

[0154] For example, an external device (such as an- 
other robot) having a similar motion language database 
can visually recognize the motion language illustrated 
in Fig. 12. Fig. 14 illustrates in the form of transitions 
from one state to another the procedures for recognizing 
the communication message "Come immediately" 
which is formed in accordance with the method used to 
realize a communication by motions having meanings 
or based on time-series changes in joint actuator an- 
gles. 

[0155] In State 30 prior to the recognition of the motion 
languages, the movement of the legged mobile robot 
100 which realizes the motion languages is visually ob- 
served using, for example, a camera. When the motion 
is recognized, the state changes to State 31 in order to 
recognize the motion language realization start motion. 
When the start motion (which corresponds to the raising 
of the right hand in the example shown in Fig. 1 2) stored 
in the motion language database, and an image input 



through the camera are compared, and do not match, 
the state returns to State 30 to perform a waiting oper- 
ation. 

[0156] On the other hand, when the motion language 
realization start motion and the image are compared 
and match, the state changes to State 32 in response 
to the completion of the reproduction of the motion lan- 
guage realization start motion. In State 32, a motion oth- 
er than the motion during the recognition of the motion 
languages is recognized. When this motion differs from 
the motion language end motion (which corresponds to 
the raising of the left hand in the example shown in Fig. 
12), the state changes to State 33. 
[0157] In State 33, the motion language database is 
searched in order to determine the motion language 
(which in this case is a movement which has a meaning) 
which corresponds to that of the recognized motion. 
[01 58] The recognized motion language is temporar- 
ily stored. The state returns to State 32 in order to exe- 
cute the operation for recognizing the motion and the 
operation for determining the motion languages. In the 
example illustrated in Fig. 12, the motion language "im- 
mediately" is determined. Then, the motion language to 
be determined is successively added to the end of the 
motion language. As a result, the communication mes- 
sage "Come immediately" which is made up of at least 
two motion languages is obtained. 
[0159] In State 32, when a motion other than the mo- 
tion during the recognition of the motion language is rec- 
ognized, and a determination is made that the motion 
corresponds to the motion language end motion (which 
corresponds to the raising of the left hand in the example 
shown in Fig. 1 2), the state changes to State 34 in order 
to set the recorded character string (that is, the motion 
languages "Come immediately" in the example shown 
in Fig. 12) as a return value, after which the state is re- 
stored to State 30. 

[0160] When the side which recognizes the motion 
languages is of the same type as the legged mobile ro- 
bot 1 00, the communication controller 232 may make a 
decision in response to the obtainment of a word which 
has a meaning as a return value. In accordance with the 
decision made, the movement controlling module 300 
may be directed to execute the motion pattern (in this 
case, running with small steps toward the side which ex- 
presses the motion languages, etc.) in response to a 
sound output or an input communication. 
[0161] The present invention has been described in 
detail with reference to a particular embodiment. How- 
ever, it is obvious that modifications and substitutions 
may be made by those skilled in the art without departing 
from the gist of the present invention. In other words, 
the present invention has been described in various 
forms for illustrative purposes only. Therefore, it is to be 
understood that the present invention is not limited 
thereto. In order to determine the gist of the present in- 
vention, one should refer to the claims of the present 
invention below. 



25 



30 



35 



40 



45 



50 



13 



25 



EP 1 136 193 A2 



26 



[0162] As described In detail above, according to the 
present invention, it is possible to provide an excellent 
or at least improved legged robot which can communi- 
cate by moving the limbs and/or the trunk, and a method 
of controlling the motion of the legged robot. 
[0163] According to the present invention, it is possi- 
ble to provide an excellent or at least improved legged 
robot which can come to an understanding with, for ex- 
ample, a robot with which a connection is not estab- 
lished through a data transmission medium, using vari- 
ous motion patterns which make use of the limbs and/ 
or the trunk, and a method of controlling the motion of 
the legged robot. 

[0164] According to the present invention, the legged 
mobile robot can communicate by moving the four limbs 
thereof. Such a form of communication by such a move- 
ment makes it possible for the. robot to communicate 
with or come to an understanding with a robot which is 
not connected through a transmission medium used for 
electronically transmitting data. 
[0165] According to the present invention, the form of 
communication based on the movement of the legged 
mobile robot makes it possible for a motion pattern itself 
which realizes the communication to realize languages. 
Therefore, even in the case where a protocol for defining 
the meaning of the motion pattern or a method of inter- 
preting it, and connection using a communications cir- 
cuit are not established between the robots, when the 
motion pattern which is executed by a first robot is vis- 
ually followed by a second robot, and an analysis based 
on the ordinary judgment of a human being which he/ 
she possesses by nature is executed, the second robot 
can guess what the first robot intended to communicate. 
[0166] According to the form of communication by a 
motion of the legged mobile robot of the present inven- 
tion, the motion pattern itself which realizes the commu- 
nication can realize languages, and the motion pattern 
which can be interpreted based on the ordinary judg- 
ment of a human being which he/she possesses by na- 
ture is used. Therefore, the robot can come to an under- 
standing with a human being in addition to a mechanical 
device which interprets electronic data, such as a robot. 
For example, a robot which has stepped into a danger- 
ous working area can visually notify, for example, the 
state of the working place to a human being who is ob- 45 
serving/watching the robot from a remote place, without 
using any data communications means, such as a wired 
or radio data communications means (or without having 
to wait for the establishment of a communications con- 
nection). 50 
[0167] Various further apsects and features of the 
present invention are defined in the appended claims. 
Combinations of features from the dependent claims 
may be combined with features of the independent 
claims as appropraite and not merely as explicitly set 55 
out in the claims. 

[0168] In so far as the embodiments of the invention 
described above are implemented, at least in part, using 



software-controlled data processing apparatus, it will be 
appreciated that a computer program providing such 
software control and a storage medium by which such 
a computer program is stored are envisaged as aspects 
s of the present invention. 



Claims 

10 1 . An articulated robot which comprises a plurality of 
joints, the articulated robot including: 

a movement controller for controlling the 
movement of each of the joints: 

f5 a motion language accumulator for accumulat- 

ing a motion language formed by a motion pat- 
tern of the joints, the motion patterns being ex- 
ecuted to express the intentions and details of 
communication; and 
20 a movement output controller for giving the mo- 

tion language to be used to the movement con- 
troller as a result of taking the motion language 
to be used from the motion language accumu- 
lator. 

25 

2. An articulated robot according to Claim 1 , further 
including an external interface used for performing 
operations including an image input operation, a 
sound input/output operation, and a data transmis- 

30 sion/reception operation through a communications 
medium, and a communication controller for con- 
trolling the communication, based on input/output 
data through the external interface, wherein the 
communication controller determines the motion 
& language to be used in accordance with the details 
of the communication. 

3. An articulated robot according to Claim 1 , wherein 
the motion language is realized using a method 
which forms a sentence by repeating the formation 
of a plurality of characters with the basic mode of 
formation being in character units, or by using a 
method which realizes the communication by a mo- 
tion which has a meaning or based on a time-series 
change in a joint angle. 

L An articulated robot according to Claim 1 , wherein 
a plurality of motion languages are combined to ex- 
press the intentions of the robot or the details of the 
communication, and wherein the motion language 
accumulator includes a motion pattern used to di- 
rect the starting of the realization of the motion lan- 
guage, and/or a motion pattern used to direct the 
ending of the realization of the motion language. 

. An articulated robot according to Claim 1 , wherein 
a plurality of motion languages are combined to ex- 
press the intentions of the robot or the details of the 
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communication, and the motion language accumu- 
lator includes amotion pattern used for directing the 
provision of a delimiter between the motion lan- 
guages. 

6. An articulated robot according to Claim 1 , further 
including a movement teaching section for directly 
teaching the motion pattern which is executed to re- 
alize the motion language as a result of directly 
holding and operating each of the joints. 

7. An articulated robot according to Claim 1 , further 
including a movement teaching section for indirectly 

. teaching the motion pattern which is executed to re- 
alize the motion language as a result of actuating 
each of the joints along a coordinate system of the 
robot. 

8. A method of controlling the motion of an articulated 
robot which comprises a plurality of joints and which 
accumulates a motion language which is formed by 
a motion pattern of each of the joints used to ex- 
press the intentions of the robot and the details of 
communication, the method comprising the steps 
of: 

determining the motion language to be used; 
and 

controlling the movement of each of the joints 
for realizing the determined motion language. 

9. A method of controlling the motion of an articulated 
robot according to Claim 8, further comprising the 
steps of inputting and outputting data to the outside 
through a communications medium, the inputting 
and outputting of data including inputting of an im- 
age, inputting/outputting of sound, and transmitting/ 
receiving of data, and of controlling communication 
based on the data input/output operation, wherein 
the motion language to be used is determined in ac- 
cordance with the details of the communication. 

10. A method of controlling the motion of an articulated 
robot according to Claim 8, wherein the motion lan- 
guage is realized using a method which forms a 
sentence by repeating the formation of a plurality of 
characters with the basic mode of formation being 
in character units, or by using a method which real- 
izes the communication by a motion which has a 
meaning or based on a time-series change in a joint 
angle. 

11. A method of controlling the motion of an articulated 
robot according to Claim 8, further comprising the 
step of executing a motion pattern used to direct the 
starting of the realization of the motion language 
and/or the step of executing the motion pattern used 
to direct the ending of the realization of the motion 



language. 

1 2. A method of controlling the motion of an articulated 
robot according to Claim 8 further comprising the 

5 step of, when the intentions of the robot or details 
of the communication are expressed by combining 
a plurality of the motion languages, executing a mo- 
tion pattern used to direct the provision of a delimiter 
between the motion languages. 

10 

13. A method of controlling the motion of an articulated 
robot according to Claim 8, further comprising the 
step of directly teaching the motion pattern which is 
executed to realize the motion language as a result 

15 of directly holding and operating each of the joints. 

14. A method of controlling the motion of an articulated 
robot according to Claim 8, further comprising the 
step of indirectly teaching the motion pattern which 

20 is executed to realize the motion language as a re- 
sult of actuating each of the joints along a coordi- 
nate system of the robot. 
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